Sequence Learning and Speech Recognition

This paper describes application possibilities for statistical methods and particulary hidden Markov models in the domain of sequence learning after treating the required basics. Especially the task of natural language processing is treated by elaborating on speech recognition and speech translation. Furthermore a network intrusion detection system to detect complex and coordinated Internet attacks is presented briefly to illustrate the universality in the concept of sequence learning. 1 Fundamentals In this section a brief overview of the needed fundamentals to understand the further discussed concepts is given and the domain of sequence learning is outlined. Further literature references are given at adequate spots to permit the consultation of more detailed background information. 1.1 Notation Overview Some syntax rules for better comprehension: • P denotes probabilities whereas p is used for probability densities. • Calligraphic letters like V are employed for sets. • Capital letters like N are utilized for constant values. 1.2 Statistical Basics Here we define stochastic processes and Markov chains – a time-restricted stochastic process. Both concepts are essential to understand the principle of hidden Markov models. Some very elementary statistical principles – that means Bayes’ rule, random variables, probability distributions and similar things are assumed to be familiar. 1.2.1 Stochastic Processes A stochastic process constitutes a series of random variables q1, q2, . . . which adopt values Qt out of a discrete or continous domain according to specific probability distributions. The distribution function of the Qt may depend on the current state qt and its predecessors q1, q2, . . . , qt−1. The stochastic process is called stationary if its behaviour is independent of the absolute value of time t. It is called causal if additionally the distribution of Qt is only a function of the previous states q1, q2, . . . , qt−1, i.e. it does not rely on states in the future. P (qt = Qt|q1 = Q1, q2 = Q2, . . . , qt−1 = Qt−1)